Knowledge-Enabled Robotic Agents for Shelf Replenishment in Cluttered Retail Environments

Autonomous robots in unstructured and dynamically changing retail environments have to master complex perception, knowledge processing, and manipulation tasks. To enable them to act competently, we propose a framework based on three core components: (◦) A background knowledge enabled perception system, which is capable of combining diverse information sources to cope with challenging conditions, such as occlusions and stacked objects with a variety of textures and shapes, (◦) Knowledge processing methods that identify the current scene, and produce strategies for tidying up supermarket racks, and (◦) The necessary careful manipulation skills in confined spaces to arrange objects in semi-accessible rack shelves. We show that our approach yields feasible, situation aware manipulation strategies. We demonstrate our framework in an idealistic simulated environment, as well as on a real shopping rack using a PR2 robot. Typical supermarket products are detected and rearranged in the retail rack, tidying up what was left after customer interaction, or while restocking sold items.